DOCUMENT RESUME 



ED 468 017 



EC 309 150 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Smith, J. David; Pare, David; Gravelle, Francine 
Alternatives to School Suspension: An Intervention for At- 
Risk Students. 

2002-04-00 

23p.; Symposium presented at the Annual Meeting of the 
American Educational Research Association (New Orleans, LA, 
April 1-5, 2002) . 

Reports - Descriptive (141) — Speeches/Meeting Papers (150) 
EDRS Price MF01/PC01 Plus Postage. 

^Aggression; Behavior Change; ^Behavior Disorders; Counseling 
Techniques; Foreign Countries; *Group Therapy; High Risk 
Students; High School Students; High Schools; Locus of 
Control; Physical Activities; ^Physical Recreation Programs; 
*Self Control; Self Management; ^Student Responsibility; 
Violence 

*Judo; ^Ontario (Ottawa) 



ABSTRACT 

This report describes the Eclipse program, which was 
developed for middle school students with school-behavior problems. The 
program combines group counseling and judo training, a non-violent sport that 
promotes social, psychological, and physical development. During the program, 
groups of approximately 15-20 students participate for 1 hour each day in 
alternating sessions of counseling and judo over a 10-week period. Since 
1999, Eclipse has been offered in 6 different schools in the Ottawa region. 

An evaluation of the program was conducted in a high school with special 
programming for youth with academic difficulties. The school had integrated 
Eclipse into their grade 9 curriculum, meaning all grade 9 students (n=50) 
were required to participate. Students with higher than average aggressive 
tendencies upon entry to Eclipse reported reductions in their aggressiveness 
at program completion. In addition, students who began the program with a 
tendency to attribute responsibility for personal experiences to external 
factors reported a shift toward taking more personal responsibility for their 
actions over the course of the Eclipse program. In contrast, students of 
average or low aggressiveness and students with an internal locus control at 
the beginning of Eclipse did not report any systematic pattern of change on 
these variables. (Contains 10 references.) (CR) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



ED 468 017 



Eclipse: Pilot Evaluation 



1 



Alternatives to School Suspension: An Intervention for At-Risk Students 

J. David Smith (Chair) and David Pare 
University of Ottawa 
Ottawa, Ontario, Canada 

Francine Gravelle 
Youth Services Bureau of Ottawa 
Ottawa, Ontario, Canada 



Symposium presentation to the annual convention of the 
American Educational Research Association 



New Orleans, Louisiana 
April, 2002 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 




U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
.y/' CENTER (ERIC) 

H This document has been reproduced as 
received from the person or organization 



originating it. 



□ Minor changes have been made to 
improve reproduction quality. 



1 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



O 

•4 J 



On 

O 

m 





2 



Eclipse: Pilot Evaluation 



2 



Program Summary 

Recent acts of extreme violence in some North American schools have focused society's 
attention on the issue of youth aggression and violence. In response, schools are increasingly 
instituting "zero-tolerance" disciplinary policies and are suspending students in rapidly 
increasing numbers. Initial research data suggest that school suspension is not an effective 
remedy for student behaviour problems. Suspension further marginalizes troubled youth and can 
lead to poorer achievement. Clearly, schools need effective alternatives to suspension to deal 
with student behavior problems. The Eclipse program was developed for middle schools students 
with school-behavior problems. The program combines group counselling and judo training. 
Group counselling captures some of the complex social dynamics that underlie anti-social 
behaviour. Judo (literally "the gentle way") is a non-violent sport that promotes social, 
psychological, and physical development. The first presenter will open this symposium with a 
multi-media presentation of the Eclipse program that explores the synergism between group 
counseling and judo training. The second and third presenters will describe the methods and 
findings of (respectively) the quantitative and qualitative components of a collaborative program 
evaluation. Finally, the discussant will facilitate dialogue on the challenges of helping at-risk 
youth through school-based programs like Eclipse in light of current school realities. 
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Alternatives to School Suspension: Introduction 
David Smith (Chair), University of Ottawa 

While recent acts of extreme violence in North American schools have defied simple and 
rationale explanation, they have also drawn society’s attention to an under-examined problem in 
our schools. The details behind these tragic events indicate that they are often the culmination of 
continuous abuse and social marginalization endured for long periods of time by vulnerable 
young people. It has become disturbingly evident that these desperate, violent acts arise from 
school environments rife with bullying and aggression. 

Schools are clearly preoccupied with youth aggression, and many use a variety of 
strategies to address the problem. Strategies can roughly be categorized as either punitive or 
remedial. Punitive measures, primarily suspension and expulsion, appear to be the most common 
strategy schools use to combat youth aggression (Roberts & Morotti, 2000). Data suggest, 
however, that these measures are largely ineffective for dealing with problem behavior in schools 
(Costenbader & Markson, 1998; Dupper, 1998). One troublesome finding is that members of 
disadvantaged groups (i.e., African and Native Americans, students from financially 
impoverished families, and students with cognitive and emotional difficulties) are over- 
represented in samples of suspended students (Morrison & D’Incau, 1997; Skiba, Peterson, & 
Williams, 1997). Clearly, schools need alternatives to punishment to deal effectively with school 
violence and bullying. 

In the last couple of decades several large-scale anti-bullying programs have been 
developed and implemented in schools, and their impact has been systematically assessed in 
outcome studies. In Norway, evaluation data on Olweus’ widely implemented anti-bullying 
program showed substantial reductions of bullying and other types of anti-social behavior as well 
as observable improvements in the social atmosphere in classrooms (Olweus, 1997). 
Unfortunately, the resounding success of Olweus’ program in Norway has not yet been 
replicated in other countries. An evaluation of a similar anti-bullying program in Toronto schools 
yielded mixed outcomes (Pepler, Craig, Ziegler, & Charach, 1994). For example, students 
reported decreasing occurrences of bullying over the evaluation period; however, more children 
were victimized based on their race, and the number of children who bullied others actually 
increased in the same period. Similarly, researchers in Belgium who studied an adapted version 
of Olweus’ program in primary and secondary schools found a pattern of positive outcomes for 
the program in the primary schools but zero outcomes in the secondary schools (Stevens, De 
Bourdeaudhuij, & Van Oost, 2000). In England, the Sheffield anti -bullying project, which is 
based on principles similar to the Norwegian program, initially showed positive outcomes in 
evaluation data (Smith & Sharp, 1994). However, a subsequent study of the longer-term impact 
of this program in four schools revealed a mixed pattern of outcomes: Two schools showed 
continuous reductions in bullying and two showed increases in bullying behaviour over the three 
year period following program implementation (Eslea & Smith, 1998). 

In summary, studies of anti-bullying and anti-violence programs show that some youth 
benefit from the interventions and others do not. Much of the research on anti-aggression 
programs focuses exclusively on program impacts. Impact studies indicate only if a program has 
achieved particular outcomes or not; typically, they reveal little about what is happening inside a 
program or why a program is effective or ineffective. Consequently, the knowledge generated by 
impact studies tends to be of limited usefulness to professionals who want to develop effective 
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interventions. The limited available data suggest, though, that who delivers a program, who 
receives a program, and how the program is implemented have significant implications for 
program effectiveness. 

Over the past year and half, we have been developing a process evaluation study the goal 
of which is to look inside a school-based anti-aggression program called Eclipse. We have been 
piloting features of the evaluation framework over the past year in preparation for full 
implementation of the evaluation study in the Fall of 2002. In this symposium we will introduce 
the Eclipse program, and we will report findings from our pilot evaluation. We will also talk 
about the challenges we have encountered in the course of our work and the adjustments we have 
made. We are very interested in engaging with those present in a discussion of the topic of youth 
aggression, how professionals can help aggressive youth, and how we can study this in a 
comprehensive and revealing manner. We are primarily concerned with conducting research that 
will be useful to people who develop and implement aggression-reduction programs for youth. 

To accomplish this, we will ground our work in the principles of action research. Action research 
is a form of scientific inquiry that addresses problems of practice and involves close 
collaboration between researchers and practitioners with common professional interests. Its goals 
are to simultaneously contribute to knowledge and improve practice, making it particularly 
suitable for investigating solutions to youth aggression. Secondly, we will study in a 
comprehensive way how the programs work and what benefits they yield. Our investigation will 
focus on program processes, specifically, on how the interventions are implemented and how 
institutional contexts influence outcomes. In our analyses we will explore the links between 
implementation and organizational variables, on the one hand, and the specific outcomes 
students evidence, on the other. We also will identify those program ingredients that have the 
greatest positive influence on outcomes. Finally, we expect to derive from these findings a set of 
principles to guide the future development of effective interventions for aggressive youth. 
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Francine Gravelle, Youth Services Bureau of Ottawa 



Eclipse program 



Background 

School violence in Canada is a major concern 
for many parents, students and teachers. A few 
schools in Ontario have implemented programs 
to combat school violence, but most lack the 
resources to support at risk or complex need 
youth. A large number of young people are not 
receiving the social skills training they need to 
help them handle conflict and power imbalances 
with peers, family members, teachers, 
employers and the community. Poorly managed 
anger is frequently a factor in assaults and other 
violent crimes that lead young people into the 
criminal justice system. Even when a student’s 
anger does not result in violence, it can lead to 
school suspension, broken family relationships, 
poor grades and other diminished opportunities if 
expressed in unconstructive ways. 

Program outline 

Eclipse is a aggression-prevention program for 
at risk youth offered jointly by the Youth Services 
Bureau of Ottawa and Judo Canada in 
partnership with the schools of the four local 
boards. The intervention program combines 
group counselling, which emphasis 
communication skills, emotional self-monitoring, 
and responsible decision making, with judo 
training, which promotes social, psychological, 
and physical development. The target clientele 
for the intervention programs middle-school 
students aged 12-15 years. Groups of 
approximately 1 5-20 students participate for one 
hour each day in alternating sessions of 
counselling and judo over a 10 weeks period. 



Program goals 

Individual outcomes 

Improves participants attendance and reduces 
aggressive behaviours. 

Help develop and maintain appropriate 
behaviours. 

Provides youth with a sense of self-control in 
the face of provocation. 

Increases self-esteem and self-control. 

Gets adolescents to participate in 
positive/recreational activities. 

Community outcomes 

Establishes community partnerships. 

Involves parents, school staff and youth 
serving agencies in the progress of the young 
people. 

As required, links students and parents to other 
services support systems. 
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Eclipse program overview 

Since 1999 Eclipse has been offered in six different 
schools in the Ottawa region both from the 
francophone and anglophone school boards. At the 
Ottawa Technical Learning Center (McArthur) Eclipse 
has been delivered to all grade nine students as part of 
their curriculum. Students get a credit for their 
participation in both components of the program. The 
counselling component is facilitated by a school staff 
and a counsellor from Youth Services Bureau for the 
duration of the program. The school staff is also 
encouraged to take part in the judo training with their 
students. The judo training is provided by a certified 
and qualified judo instructor referred by Judo Canada. 

Partners of Eclipse 

Youth Services Bureau 

• In partnership with the four school boards identify 
school to host the program 

• Provide trained Eclipse staff 

• Provide training to school personnel for 
counselling component 

• Manage the program 

• Follow-up with participants and parents as 
required. 

School boards 

• Provide credit to students enrolled in Eclipse 

• Provide a teacher/guidance counsellor to 
participate in the program delivery 

• Provide a space for the counselling groups and the 
“dojo” for the judo training. 

• Help with project promotion. 

Judo Canada 

• Train and ensure adequate supply of judo 
instructors. 

• Promote the project 

• Participate in the steering committee 



Partners of Eclipse con’t 

University of Ottawa 

• Develop an evaluation framework and data 
collection tools 

• Supervise data collection and entry. 

• Analyze results and present reports to Eclipse 
staff and the steering committee. 

Program feedback 

" The program is teaching me that there are 
other ways to express myself when I get 
angry. I walk away from a situation if I’m 
angry and give myself time to calm down and 
then I can deal with the issue. ’’(Eclipse 
participant) 

“’This is definitely a positive and valuable 
resource for our students. In fact, we NEED 
this program.’’ (Eclipse Principal) 

‘He used to answer back rough and now it’s 
completely different, he knows how to answer 
back without yelling or talking back, he used 
to keep too much inside and now he is able to 
talk to us about it. ..a lot of things. ” (Comment 
from a parent) 

The cost per participant 

The cost per participant per week to take part in the 
Program is 31 .41 $. The uniforms are provided for the 
duration of the program. 
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Evaluating the Eclipse Program: Quantitative Methods and Findings 
David Smith, University of Ottawa 



Method 

Instruments. The process of selecting instruments for this evaluation study began with a 
discussion with program developers on what they believed were the primary anticipated 
outcomes for Eclipse. Ultimately, two constructs were selected for examination in the 
quantitative component of the evaluation: aggression and locus of control. We then identified 
several possible measures that were well established standardized instruments (with norms) and 
are widely used in similar research. Program personnel vetted the instruments for 
appropriateness, particularly in terms (a) their length — they had to be short — and (b) language — 
they had to be readable by adolescent participants, many of whom had significant reading 
difficulties. In the end the following two instruments were selected: 

1. The Nowicki-Strickland Locus of Control Scale (NSLC) is a self-report instrument that 
measures whether individuals tend to attribute their experiences to their own decisions and 
actions (internal locus of control) or to factors outside of their control (external locus of 
control). For this study, we used the 19-item version for children in grades 7-12. 

2. The Aggression Questionnaire (AQ) is a self-report instrument that measures respondents’ 
aggressive tendencies in five different domains: physical aggression, verbal aggression, 
anger, hostility, and indirect aggression. We used the 15-item research version of the 
instrument for this study. 

To broaden the scope of our evaluation of program outcomes, we are also collecting information 
on school attendance and office disciplinary referrals. Data on these indices are collected for all 
student participants beginning three-months prior to their starting the program and ending three- 
months post-program. Because of the length of this timeframe (and, additionally, because of 
recent procedural changes in how these data are tracked in the school), we do not have sufficient 
information to report findings at this time. 

Procedure. The two questionnaires, the AQ and the NSLC, are administered twice during 
each program rotation: once at the beginning of the program and approximately 10 weeks later at 
the termination of the program. All data on these measures are analyzed in light of determining 
the amount and direction of change from pre-teat to post-test on the constructs. 

Adjustments to Method 

We piloted this evaluation in a high school with special programming for youth who have a record of 
academic difficulties. The school had integrated Eclipse into their grade 9 curriculum, meaning that 
all grade 9 students are required to participate in the program (although participation in the 
evaluation study is optional). At data collection sessions for early program rotations involved in this 
study, we observed the following: 

• A large minority of students were very slow in completing the questionnaires, taking up to a full 
hour to complete the questionnaires (totaling 34 single-sentence items). 

• Many students required substantial individual help to understand the items. 

• A few individuals known to program staff to have significant reading problems did not ask for 
help. 

• Many completed questionnaires had one or more blank items. 

• Some questionnaires were returned blank. 
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These observations led us to be concerned about whether or not student-participants were adequately 
comprehending questionnaire items. We made several adjustments to enhance students’ 
comprehension of the questionnaires, including the following: 

• We altered the visual presentation of questionnaires, making them simpler and reducing potential 
distractions (e.g., additional items on the AQ). We enlarged the print so it was easy to read. 

• A counsellor now reads the questionnaire items aloud one-by-one while students follow along on 
the form and mark their responses. 

• Other program personnel are present in the classroom to provide individual assistance to students 
when required. 

Recent reports from the counsellors who administer the instruments suggest that the effects of these 
changes are positive: students seem to be completing the questionnaires without difficulty. 

Results 

NSLC measures taken just prior to the start of Eclipse show that the female and male 
participants in this study had a more internally oriented locus of control compared to the norm 
group: Girls (N = 14) scored 10.3 compared to norm of 12.3 and boys (N=35) scored 9.1 
compared to norm of 13.8. On the AQ, overall aggression scores for boys and girls in our sample 
were squarely average compared to youth in the general population. The mean T-score was 50 
for girls and 51 for boys. Boys and girls scores on the NSLC and the AQ did not differ 
significantly, so data from both gender groups are combined in all subsequent analyses. 

It was predicted at the outset of this study that students who complete the program would 
learn to take more personal responsibility for their behaviour and become less aggressive. We 
expected that these outcomes would be manifested in lower aggression scores and lower (i.e., 
more internal) locus of control scores from pre-test to post-test. Our results do not support these 
hypotheses when the sample data is examined as a whole. On the NSLC, the pre-test scores did 
not differ significantly from post-test scores, t{ 43) = 1.7, p = .10. Similarly on the AQ, pre-test 
scores for the sample did not differ significantly from post-test scores, t(42) = .90, p = .38. 

Given the original mandate of the Eclipse program to help aggressive youth and the fact 
that our participants as a group did not fit this profile, we decided to split the sample into groups 
based on levels of aggression and locus of control at intake and then make pre-test/post-test 
comparisons across sub-groups within each of these variables. We hypothesized that students 
resembling the profile of clientele for which Eclipse was initially developed (i.e., aggressive, 
externally-oriented youth) would show reductions on these measures and the remaining students 
not resembling this profile would show no change. To this end, two locus-of-control groups were 
created: an External group (those who scored above the NSLC scale mean) and an Internal group 
(those who scored below the mean). The Internal group showed no change from pre-test to post- 
test on the NSLC, t( 38) = .07, p = .94. The External group, however, showed a significant 
change from pre-test (M = 13.3, SD = .5) to post-test (M = 8.3, SD = 1.9), t( 5) = 7.3, p = .001, 
and External group’s mean score was virtually identical to the Internal group’s score post- 
program. The sample was similarly divided into three aggression groups (score variability 
permitted a larger number of groups): Low (T-scores of 44 and lower), Average (45-55), and 
High (56 and above). Results revealed that students in the High group showed a significant 
reduction in aggression from pre-test ( M = 60, SD = 4.7) to post-test (M = 56, SD = 7.4), t( 12) = 
2.1, p = .06, whereas the students in the Average and Low groups showed no significant changes 
in aggressiveness from pre-test to post-test, ts(22, 8) = .20 and -.85 respectively, ps > .05. 
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Conclusion 

Students who reported having higher than average aggressive tendencies upon entry to 
Eclipse reported reductions in their aggressiveness at program completion. Additionally, students 
who began the program with a tendency to attribute responsibility for personal experiences to 
external factors reported a shift toward taking more personal responsibility for their actions over 
the course of the Eclipse program. In contrast, students of average or low aggressiveness and 
students with an internal locus control at the beginning of Eclipse did not report any systematic 
pattern of change on these variables. These students appeared to be unaffected by Eclipse with 
respect to locus of control and aggression. We interpret this as a positive finding. It seems 
reasonable to assume that life circumstances sometimes require aggressive responses and 
external attributions, and the goal of programs like Eclipse should not be to eliminate these 
behaviours but to reduce them to appropriate levels. Additionally, these results should not be 
interpreted as meaning that Eclipse does not benefit non-aggressive youth. It is likely that these 
students benefited from Eclipse in ways that were not detected by our two instruments. The 
finding has led us to consider adding another instrument to our battery, which would permit us to 
capture a broader range of program outcomes. Additionally, school attendance and discipline 
referral data will provide valuable information on these important school-related outcomes. This 
data should become available to us for analyses in the very near future. 

We remind readers that data on Eclipse are preliminary, and our findings must be 
interpreted with caution for several reasons. In the first place, the number of participants is small, 
and this reduces the generalizability of these findings to other groups of students. Clearly we 
need more data to verify these initial results and to identify the dimensions of Eclipse that 
contribute most to positive outcomes. Secondly, we did not use a control group, which raises the 
possibility that quantitative findings represent a regression-to-the-mean effect. We believe that 
this is not a strong possibility, though, since we observed this pattern of change in high-scoring 
groups but not low-scoring groups, as would be expected. Finally, as this evaluation is still in a 
pilot phase, we have made many small adjustments to our methods over the course of data 
collection. Although we do not believe that these adjustment markedly altered our results, this 
assumption can only be verified through more data collection. 
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Evaluating the Eclipse Program: Qualitative Methods and Findings 
David Pae, University of Ottawa 

The qualitative component of our evaluation of the Eclipse program includes three 
primary data sets: 

1. Individual exit interviews with student participants 

2. Focus group discussions with program delivery personnel 

3. Post-program phone interviews with parents 

Individual Exit Interviews with Student Participants 

Individual, semi-structured exit interviews were conducted with all participants in the 
Eclipse program. The interview guide was altered part way into the project to accommodate for 
response patters (see Appendix A for interview formats). We found that the initial round of 
questions called on students to reflect on personal changes too abruptly in the interviews, and 
that the questions also presupposed an inordinately high level of familiarity with the instrumental 
objectives of the program in general. In other words, it is hard to answer “what is helpful” when 
one doesn’t have a clear idea of what in particular, “needs help”. This dilemma is related to the 
issue cited in the summary of the quantitative findings; our population was not a homogenous 
group clearly being targeted for violent and/or aggressive behaviour. The second set of questions 
therefore elicits student likes/dislikes regarding judo and group, rather than calling upon them to 
evaluate the efficacy of the program components. 

Once PaEand research assistant Gignac had developed a protocol for summarizing each 
interview and including transcribed students quotes, summary sheets were completed for all exit 
interviews. In analysing these interview data, we distilled participant comments into three 
response domains: 1. Judo 2. Group 3. Personal Changes. Gignac then engaged in inductive 
content analysis, coding student statements under these three domains and assigning descriptors 
to clusters of responses. PaE subsequently examined all coded statements and descriptors, making 
corrections and modifications in conjunction with Gignac. Some statement descriptors were 
eliminated, others were added, and adjustments were made to the wording to more closely reflect 
the content of participant statements. Approximately 25 response descriptors were generated for 
each of the three response domains (see Appendix B). 

PaEand Gignac independently assigned each of these descriptors a valence of either 
positive (P) or negative (N). With respect to personal changes, positive responses were those 
gauged to reflect personally beneficial or pro-social changes, while negative responses were seen 
as speaking of personally detrimental or anti-social changes. Assignment of P and N valences 
was identical between the two independent raters. The third column in each of the three response 
domain charts in Appendix B indicates the number of respondents making comments under each 
response descriptor. 

The subsequent step in data analysis involved the development of higher order response 
themes. Working in tandem, PaEand Gignac distilled the many response descriptors into higher 
order themes, maintaining the positive/negative criterion. (See Appendix C for these distilled 
themes and their frequency.) Generally speaking, interviewees responded with short statements 
and did not elaborate at length on their answers. The relative brevity of the interviews (10 to 15 
minutes) was a constraining factor. In addition, the students enrolled in the target schools, by 
definition, experience academic challenges and are not typically verbally oriented. For these 
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reasons, much of the statements recorded and tabulated are brief and relatively unadorned. In the 
section below, we provide a number of comments which are more developed than the average 
response from participants. 

Positive responses to the Judo outnumbered negative comments by a ration of about three 
to one. The favourable comments centred on four themes: 

1. Entertaining Diversion - Many respondents described their appreciation of the Judo as an 
activity that contrasted with the usual school routine, without actually pointing to a specific 
‘benefit’ from the activity itself. Comments here centred on the ‘fun’ of flipping people 
(particularly school staff) and engaging in various Judo games. 

2. Mastery/Skill Development - A number of respondents indicating they appreciated the 
opportunity to learn judo moves, above and beyond merely having fun engaging in the judo. 
Many expressed satisfaction at having attained a degree of proficiency in a skill. 

“In judo, when I was fighting, I'm like ‘I don ? want to do that, it's for guys’, but when I tried 
it, I hi like ‘Yeah, its good, I want to learn some moves. ’ ...it said I had muscles, I could 
defend myself, I could defend someone that ’s like in trouble and help them out. ” 

“I liked learning from Nathalie the Judo moves, the techniques. It made me feel good. ” 

“I liked the throwing part, I was very good at doing that. ” 

3. Physical Benefits - Another cluster of remarks about the judo centred on its physical benefits: 
students said they liked being active; they felt more awake, or relaxed, and the judo gave 
them a healthy appetite. 

4. Social Benefits - In some cases, judo was cited as a context for sharing an enjoyable 
experience with others, being surrounded by “happy faces”. 

Negative responses to the judo centred on : 

1. Disruptions/Consequences - It appears that, despite the focus on self-discipline and mutual 
respect, judo also provided a forum for some students to act out. A number of participants 
complained about disruptive behaviour by certain students, and of the time out (“moxo”) 
which was the consequence assigned for this behaviour. 

2. Physicality/Intimidation - The judo intimidated some students, who complained about 
having, at times, to go up against larger or more aggressive opponents. Some participants 
complained about rug bums or the physical pain that accompanies being thrown. One student 
pointed to what they saw as an apparent contradiction between judo’s emphasis on self- 
discipline but its apparent focus on aggression. 

“I don ‘t like getting hurt. They whacked me, it was embarrassing. ” 

“Its conflicting— fighting and judo and learning self-discipline in the same hour.. .I’ve heard 
some kids, grade 6 boys, outside that have been using their judo moves against littler 
children, helpless children. And I think its really conflicting because self-discipline teaches 
you not to fight, judo teaches you to fight. ” 

3. Miscellaneous - A small handful of other negative comments surfaced. These included a 
dislike for the judo uniforms, a complaint about being more tired as a result of the activity, 
the wish for a completion award, a distaste for the exercises, and the wish that there had been 
more time set aside for the judo. 

Like judo, the groups were described in positive terms about three times as frequently as 
they were critiqued. Positive responses to the groups centred on: 

1. Sharing Feelings/Experience - A large number of the favourable comments on the group 
process centred on the opportunity to share one’s experience and to hear the experience of 
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others. It is worth noting that this verbal sharing process was also targeted as a negative 
feature by another, smaller group of respondents. 

“I liked whenever he would bring up the topic and we would have to answer... .like if he said 
‘what do you feel like when you want to fight?’, he 'd point to you and you had to like tell him 
...you had a chance to talk to someone ... ” 

“It’s nice to talk about things; its fun. Everyone had a chance to talk about the different stuff 
that they do and I liked listening to them... we talk about our problems and other people are 
giving advice... ” 

“It helped you figure out what your feelings are. ..hearing what other people have to say 
about themselves, being able to tell other people my feelings. ” 

2. Structured Activities - The structured activities varied in the groups — what links them is that 
they represent an alternative to the group verbal sharing. A number of participants spoke 
favourably of the role-plays. Other, rarer comments centred on the writing activities, skits, 
movies, and the treats, food and music. 

3. (writing activities): “Sometimes you can ‘t say something but you can always write it down or 
something.. ’’. 

4. Developing Pro-Social Skills - A small number of respondents spoke favourably of the 
opportunity to develop social skills: learning to be more verbally assertive and socially 
respectful, learning about self talk. 

Negative responses to the groups were varied; the one recognizable cluster of responses was 
centred on the same theme that dominated in the positive responses: the group sharing process. 
Other negative comments were difficult to group thematically and are listed as miscellaneous. 

1 . Group Sharing - Some students complained of having no choice about reflecting aloud or 
not; some said it was difficult to talk about issues. Others said there was too much talking, 
not enough interactive games, and that the topics were redundant after a while. 

“I don t like sitting and talking about that stuff... I learned new words in there, like ‘self- 
esteem ’, but that ’s not for me. ” 

2. Miscellaneous - As with the judo, there were a few miscellaneous comments that didn’t 
exhibit any obvious pattern. These included complaints about too much paperwork, boredom 
with the process, and a dislike for role-playing. 

“He gave us paper work that wasn’t really necessary.. .like maybe to him it was but not to 
us... we all disagreed with it but he just kept on going on with it...what’s that teaching us? It 
really wasn ’t teaching us anything. ” 

While favourable outnumbered unfavourable comments by abut three to one for both the 
judo and the groups, responses regarding personal changes were even more imbalanced. Students 
were virtually unanimous in citing changes judged to be personally beneficial and pro-social (as 
opposed to personally detrimental and anti-social). These positively-oriented comments were 
distilled into four categories: 

1. Increased Self Determination/ Composure - A number of students indicated the program had 
taught them to control their anger more effectively, that they had learned to make choices and 
exercise judgment, to be less reactive and more in control, and to attend better to thoughts 
prior to acting. 

“I can control my anger better.. .every time someone got me mad I could control it.. .before I 
joined the Eclipse program I would get so angry I couldn ’t control... (now) I would, um, well, 
if someone called me a name, I’d say ok if that 's what you think... that 's your opinion but 
that ’s not what I think myself... ” 
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“Whenever I feel like 1 am mad 1 just take a walk and calm down, talk to someone, like 
that... like 1 just count to ten and if that doesn’t work just ask the teacher to leave the class, 
walk around the school. 

2. Social Ease/ Adeptness - A second cluster of responses about changes centred on improved 
social skills in a variety of domains: students indicated they were acting with more respect, 
listening better and trusting more, connecting with and appreciating people more, expressing 
feelings, being helpful at home and generally more cooperative, and acting with more social 
assertiveness sand confidence. 

“I act better. If a person is feeling down I try to boost their confidence up. ” 

3. School - Comments related to changes at school included references to increased 
concentration, less trouble, an improvement in marks, and a more positive attitude abut 
attending school. 

4. Well Being - This higher order theme included comments related to improved disposition, 
reduction in feelings of anger, improved self-image, and an increase in physical activity. 

“[I’m more] positive now, and it’s better now at home school, camp, friends, different 
parents, mother and brothers have noticed... ‘ How are you doing this to yourself? You have 
improved; you were out of your mind last year. What did you eat?’ They are surprised. ” 

“It helped me with my physical activity... in gym I’m more active now. ” 

A minority of students indicated they weren’t aware of any changes in themselves during 
the duration of the program. Of the 49 students included in this data set, only one student 
indicated a change judged to be anti-social and personally detrimental. That student spoke of 
becoming more aggressive as a result of the judo training. 

“My friend used to just play box with me and I wouldn ’t really hit back hard but now he notices 
that 1 am more aggressive ... he 's like you ’re going to hurt me ...and I’m like good its about time, 
so now we don t just fool around but we actually get into fights. ” 

Discussion and Limitations. Although limited in verbal richness, the exit interview data 
seem to clearly suggest that students found much that was attractive in both the judo and group 
components of the Eclipse program. Some of the appeal appears to simply be a function of the 
disruption of ordinary school routines; however many of the comments touched on components 
and features of the program deliberately focused on Eclipse’s therapeutic and educational 
mandate. While the concerns expressed about the group work tend to be heterogeneous and do 
not suggest much in the way of adjustments, the comments on the judo exhibit more of a pattern. 
For a number of students, the judo was an intimidating experience, and this likely diminished the 
benefits they might have derived from it. It may be that some accommodations could be made to 
the delivery of the judo that would reduce these concerns; for instance students might be given 
more choice about the scope of the moves they engage in, and perhaps the size of their 
opponents. Of course it would be important that these adjustment be made in a manner that does 
not violate the spirit and tradition of judo itself, and it would be critical that judo consultants play 
a central role in modifying the program to accommodate student concerns. 

The findings on perceived changes are particularly striking. While it is understood that 
students may have felt mandated to report on ‘favourable’ developments, they were not 
prompted in any way to do so. No doubt the program components — particularly the group — 
furnished them with new words for describing their experience, and this may have inspired the 
use of more jargon than they would have produced otherwise. Nevertheless, virtually all 
students interviewed, if they reported no changes, spoke of alterations in their behaviour and 
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moods that were judged to be personally beneficial and pro-social. While, as explained above, 
the population engaged in Eclipse over the period studied does not precisely match the profile of 
the aggressive student for whom the program was developed, many of the participants spoke of 
changes associated with reducing aggression and developing self control. It is of course difficult 
to gauge from their comments how perceptible the changes participants report are to an outside 
observer, their reflections alone suggest the development of a vocabulary that will serve the 
students as they continue to address issues related to self-regulation, social interaction, and so on. 

A limitation to the data from the student exit interviews relates to verbal richness: the 
frequently terse responses do not furnish a rich and nuanced picture of their engagement with the 
program. A possibility currently being considered is to interview students in a focus group 
format so that they might “cross-fertilize” each other as they respond to questions. However, this 
approach could also be compromised by a degree of ‘groupthink’ that is an inevitable byproduct 
of group interviewing procedures. 

Focus Group Discussions With Program Delivery Personnel 

The researchers met with Youth Services Bureau, Judo Canada, and school personnel 
involved in delivery of the Eclipse program halfway into delivery of the program in the spring of 
2001. We utilized a focus group format to explore various aspects of the program. Participants 
were asked to comment on both the positive features and the challenges of each of the two 
Eclipse components, the judo sessions and the small group work, and to reflect on the perceived 
overall impact of the program. What follows are some general findings based on the transcribed 
session and additional field notes. 

Judo. It was widely agreed that the judo component was particularly effective. Students 
were seen to interact in largely pro-social ways within the highly structured framework of the 
judo sessions. Several participants commented that the judo promotes an atmosphere of mutual 
respect and self-control which are inherent features of the ancient tradition. The mat was seen as 
a place where the rules of regular life were suspended, competition was set aside (or at least 
tempered), and students adhered to the judo code. There was some talk of expanding the 
ritualized nature of the judo sessions into the group component of the program. However, it was 
found, especially with the younger group, that some students were wary of physical injury. It was 
also felt that the longer judo sessions (at OTLC) were more successful than those held in a 
shorter time frame. Finally, there was speculation about the importance of providing some 
ongoing judo involvement as a follow-up so that the skills and attitudes gained could be 
maintained. 

Groups. Trust was seen to build in the groups over time. The program was seen as a 
potential link to additional YSB services — that is, by making a connection with YSB staff, it was 
felt some students might feel more free to pursue further services offered by the agency. The 
YSB group facilitators shared ideas with school personnel and modified the group delivery over 
time. Students appeared to favor experiential exercises that contrasted with didactic classroom 
activities. The ritualized order of the judo sessions did not carry over to the groups, and it was 
felt the groups would benefit from tighter structures and guidelines for behavior. The course 
material was less easily adapted for the younger students at St. Michael’s than for the older 
population at OTLC. 

Overall Program Impact. The Eclipse program at OTLC has a high profile and positive 
reputation amongst staff and students. OTLC students ask when their rotation starts; students 
from other schools have expressed an interest in getting a similar program. OTLC teaching staff 
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not currently involved with the program have expressed an interest in being involved. The 
preliminary observation from the school staff involved in program delivery is that there has been 
a noticeable decrease in volatile behavior and inappropriate language at the school since the 
initiation of the program. St. Michael’s staff also indicated the program has been markedly 
positive in its impact at as well; they suggested it would also be beneficial for the girls there. 

In terms of challenges faced, personnel involved in service delivery recommended a 
streamlining of paperwork and data gathering processes that, at the time, were somewhat 
duplicated between YSB and University of Ottawa stakeholders. There were also concerns that 
communication between the various stakeholders should be more regular. Since the time of this 
focus group, adjustments have been in these and other areas. 

Discussion and Limitations. It is interesting to note that the service deliverers more 
uniformly perceived the judo as beneficial, despite that fact that a large number of students also 
referred to benefits derived from the group process. One reason for this may be that engagement 
with group processes is more clearly variable — from highly self-disclosing and involved to 
reticent and peripheral — whereas with the judo, participants are either “in” or “out” (moxo). It 
also appears that students were more overtly enthused about the judo, perhaps creating the 
impression that it was more successful. It is likely not ‘cool’ to be enthused about group sharing 
processes, and thus students may have appeared to benefitting from the group less than they later 
(privately) said they were. 

A difficulty in gauging the impact of the program on students from the focus groups has 
to do with the investment in the program on the participants. The staff involved in Eclipse from 
the schools, YSB, and Judo Canada are all enthusiastic about the program and are probably more 
inclined to notice and report on positive developments for that reason. A useful additional 
measure, though one that presents considerable logistical challenges, would be the perspective of 
school personnel with less of a stake in Eclipse. 

Post-Program Phone Interviews With Parents 

Audiotaped phone interviews were completed with parents after their sons/daughters 
competed the program. Because phone interview data gathering is not yet complete, we will here 
present only some general preliminary findings. Two questions were posed: 

1. Please describe what you know about the Eclipse Program? 

2. Have you noticed any changes in your child since they started the program? 

Of the parents contacted to date, a much higher percentage (more than four-fifths) 
indicated some knowledge of the judo than the group component (about a third). It is not clear if 
this is related to how the program was introduced to parents. It seems fair to assume that part of 
the reason for this is that the world “judo’ tends to evoke a fairly singular image to most people, 
whereas “group” is an ambiguous word, and group counselling or psychoeducational processes 
are less familiar to the general public. In addition, when students speak to their parents about the 
program, talk of ‘flipping’ classmates on a mat likely stands out more than references to group 
discussions and role-plays, which might be construed as part of the regular school curriculum. 

Just over two-thirds of the parents reached indicated they saw change of some sort in 
their children. As in the student’s self reports, these changes were almost uniformly positive. 

One parent reported increased moodiness, but then volunteered this was likely more a reflection 
of adolescent hormonal changes than the program’s impact. In addition, there were two 
comments about the child feeling more tired; but it wasn’t clear if this was gauged to be a bad 
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thing. Positive comments about changes observed are displayed below as higher order themes 
with subcategories of responses. Because the interviews are still in process, we have elected not 
to include frequency data at this time. 

1. Pro-social behaviour - more self control; calmer; fewer angry outbursts, less reactivity; 
transfer of learned positive attitudes to other contexts (eg. School work, home); more 
responsibility. 

2. Personal development - more physically active; more autonomous, greater maturity; more 
optimistic; more enthusiasm; more mastery; increased self confidence; recognizing and 
expressing feelings more, better concentration. 

3. Other - More health conscious; using judo skills elsewhere to avoid fighting. 

Discussion and Limitations. It is perhaps not surprising that a number of parents showed 
limited familiarity with the Eclipse program; at the best of times in today’s busy world, parents 
have difficulty monitoring their children’s’ school activities closely. Due to the various 
academic challenges faced by the student participants in the Eclipse program, they may be 
somewhat more likely to be aberrant in transporting materials and communicating information to 
their parents. 

The data from parents is incomplete due to the unreliability of a phone interview protocol 
that is dependent on people being available during those periods when the researcher is gathering 
data. At this time the parents’ view helps to paint a fuller picture of Eclipse, but is lacking in 
breadth. It also falls short of the depth offered by a face-to-face interview; however without 
significantly more person-power, such an interview format would prove unwieldy and might 
compromise, rather than improve, the scope of data gathered. 
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Appendix A 

Student Exit Interview Guides 



Groups 1-3 

1 . What, if anything, about you has changed as a result of the program? 

2. What part of the program did you find most helpful? 

3. What part of the program did you find least helpful? 

4. What, if anything, would you change about the program? 

Groups 4-6 

1. What was it like to be in the Eclipse program? 

2. If you had to share one experience from the Eclipse program that you had - good or not - what 

would that be? 

3. What did you like/ dislike about the Judo part? 

4. What did you like/ dislike about the Group part? 

5. What changes in yourself because of Eclipse? 

6. Would you make any changes to the program? 
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Appendix B 

Student Exit Interview Response Domains and Descriptors 



Judo 



Response Descriptor 


Valence 


Frequency 


1 . Liked flipping people/kicking 


P 


9 


2. being active 


P 


1 


3. Learned self-defence 


P 


6 


4. Makes me wake up/good for me 


P 


2 


5. Liked the moves/techniques 


P 


6 


6. Liked the way judo was taught 


P 


1 


7. Able to do it/mastery 


P 


4 


8. Did not like consequences (“moxo”) 


N 


5 


9. Made me hungry 


P 


1 


10. Liked judo games 


P 


3 


1 1 . Liked the exercises 


P 


2 


12. Didn’t like fighting/getting hurt 


N 


2 


13. Didn’t like uniforms 


N 


2 


14. Didn’t like disruptive behaviour 


N 


2 


16. Found judo fun 


P 


7 


17. It was relaxing 


P 


2 


18. Power imbalanced favoured opponents 


N 


1 


19. Rug bums and physical pain 


N 


3 


20. Insufficient time scheduled 


N 


4 


21. Enjoyed social contact 


P 


3 


22. Liked the fighting and competitions 


P 


5 


23. Feel more tired as a result 


N 


1 


24. Didn’t like exercises 


N 


1 


25. Judo is oriented towards aggression 


N 


1 



Group 



Response Descriptor 


Valence 


Frequency 


1 . Liked doing role-plays 


P 


8 


2. Liked treats, food, music 


P 


6 


3. Not enough interactive games 


N 


2 


4. Liked check-ins 


P 


2 


5. Liked talking about problems 


P 


5 


6. Learning that others have problems 


P 


6 


7. Didn’t like role-plays 


N 


3 


8. Too much talking 


N 


1 


9. Pace too quick 


N 


1 


11. Topics redundant 


N 


3 


12. Disliked paper work 


N 


4 


13. Liked expressing feelings 


P 


5 


14. learning to be assertive and respectful 


P 


2 
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15. Watching movies 


P 


3 


16. Became bored/tired/hungry 


N 


2 


17. Liked personal sharing 


P 


1 


18. Liked participating in skits 


P 


4 


20. Chance to talk and be heard 


P 


5 


21. Difficult talking about things 


N 


2 


22. Writing activities 


P 


1 


23. Talking about anger 


P 


2 


24. Anger management was too obvious 


N 


1 



Personal Changes 



Response Descriptor 


Valence 


Frequency 


1 . Controlling anger/temper better 


P 


12 


2. Learned to be respectful/supportive 


P 


2 


3. Listens better now 


P 


1 


5. Learning to make choices 


P 


,3 


6. Nothing has changed 




3 


7. Increased concentration 


P 


1 


8. More connecting with/appreciating people 


P 


1 


9. Feeling more positive/happy 


P 


1 


10. Better judgment 


P 


6 


1 1 . Expressing/coping with feelings better 


P 


3 


12. Less reactive, more self control 


P 


5 


13. Getting in less trouble at school 


P 


4 


14. Marks improving 


P 


1 


15. More helpful at home 


P 


1 


16. More self confident/assertive 


P 


3 


17. Less angry 


P 


1 


18. More cooperative 


P 


2 


19. More aggressive 


N 


1 


20. More reflective, attentive to thoughts 


P 


1 


21. More positive about coming to school 


P 


1 


22. More active 


P 


2 


23. More positive about self 


P 


1 
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Appendix C 

Student Exit Interviews: Higher Order Response Themes 



Judo: Positive 



Higher Order Theme 


Frequency 


Higher Order Theme 


Frequency 


Entertaining Diversion 
Liked flipping people/kicking 
Liked Judo games 
Found judo fun 


19 


Masterv/Skill Development 
Learned self defence 
Liked moves/techniques 
The way it was taught 
Able to do it/mastery 
Liked fighting/competitions 


22 


Physical Benefits 
Being active 

Makes me wake up/good for me 
Liked exercises 
It was relaxing 
Made me hungry 


8 


Social Benefits 
Enjoyed social contact 


3 



Judo: Negative 



Higher Order Theme 


Frequency 


Higher Order Theme 


Frequency 


Disruptions/Consequences 
Didn’t like consequences (‘moxo’) 
Didn’t like disruptive behaviour 


7 


Phvsicalitv/Intimidation 
Fighting/getting hurt 
Power favoured opponents 
Rug burns/physical pain 
Judo oriented towards aggression 


7 


Miscellaneous 
Didn’t like uniforms 
Wanted a completion award 
Insufficient time scheduled 
Feel more tired as result 
Didn’t like exercises 


8 







Group: Positive 



Higher Order Theme 


Frequency 


Higher Order Theme 


Frequency 


Sharing Feelings/Experience 
Liked check-ins 
Liked talking about problems 
Learning that others have problems 
Liked expressing feelings 
Liked personal sharing 
Chance to talk and be heard 
Talking about anger 


26 


Structured Activities 
Liked role-plays 
Liked writing activities 
Liked participating in skits 
Liked treats, food, music 
Watching movies 


22 


Developing Pro-Social Skills 
Learning to be assertive/respectful 
Learned about self-talk 


3 
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Groups: Negative 



Higher Order Theme 


Frequency 


Higher Order Theme 


Frequency 


Group Sharing 
Too much talking 
No choice about reflecting 
Not enough interactive games 
Topics redundant 
Difficult talking about things 


8 


Miscellaneous 
Pace too quick 
Didn’t like role-plays 
Disliked paperwork 
Became bored/tired/hungry 
Anger management was too obvious 


11 



Personal Changes: Positive 



Higher Order Theme 


Frequency 


Higher Order Theme 


Frequency 


Increased Self Determination/ 
Composure 

Controlling anger/temper better 
Learning to make choices 
Better judgment 
Less reactive/more self-control 
More reflective/attentive to thoughts 


27 


Social Ease/Adentness 
More respectful/supportive 
Listens better now 
Trusts people more 
Connecting with/appreciating people 
Expressing/coping with feelings better 
More helpful at home 
More self confident/assertive 
More cooperative 


13 


School 

Increased concentration 
Getting in less trouble at school 
Marks improving 

More positive about coming to school 


7 


Well-Being 
Feel more positive/happy 
Less angry 
More active 

More positive about self 


5 



Personal Changes: Negative and Neutral 



Higher Order Theme 


Frequency 


Higher Order Theme 


Frequency 


Aggression 
More aggressive 


1 


Absence of Impact (Neutral) 
Nothing has changed 


3 
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